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ABSTRACT 

The United States Training and Employment Service 
General Aptitude Test Battery (GATB) , first published in 1947, has 
been included in a continuing program of research to validate the 
tests against success in many different occupations. The GATB 
consists of 12 tests which measure nine aptitudes; General learning 
Ability; Verbal Aptitude; Numerical Aptitude; Spatial Aptitude; Form 
Perception; Clerical Perception; Motor Coordination; Finger 
Dexterity; and Manual Dexterity. The aptitude scores are standard 
scores with 100 as the average for the general working population, 
and a standard deviation of 2 0. Occupational norms are established in 
terms of minimum qualifying scores for each of the significant 
aptitude measures which, when combined, predict, job performance. 
Cutting scores are set only for those aptitudes which aid in 
predicting the performance of the job duties of the experimental 
sample. The GATB norms described are appropriate only for jobs with 
content similar to that shown in the job description presented in 
this report. A description of the validation sample and a personnel 
evaluation form are also included. (AG) 



ED 066476 



March 1968 



United States Employment Service Technical Report 



S-314 




^ 231 ^ 



Development of USES Aptitude Test Battery 

for 



o 

o 



Programmer, Business 

(profess. & kin.) 020.188 




U.S. DEPARTMENT OF LABOR 

MANPOWER ADMINISTRATION 




1 



S-314 



Technical Report on Development of USES Aptitude Test Battery 
For . • • • 

Programmer, Business (profess. S kin.) 020.188 

S-314 

(Developed in Cooperation with the California, 

Ohio and Wisconsin State Employment Services) 



U.S. DEPARTMENT OF LABOR BUREAU OF EMPLOYMENT SECURITY 

Willard Wirtz, Secretary Robert C. Goodwin, Administrator 



MANPOWER ADMINISTRATION 
Stanley H. Ruttenberg 
Administrator 



U.S. EMPLOYMENT SERVICE 
Charles E. Odell, 
Director 



March 1968 



FOREWORD 



The United States Employment Service General Aptitude Test Battery 
(GATB) was first published in 1947, Since that time the GATB has 
been included in a continuing program of research to validate the 
tests against success in many different occupations. Because of its 
extensive research base the GATB has come to be recognized as the 
best validated multiple aptitude test battery in existence for use 
in vocational guidance. 

The GATB consists of 12 tests which measure 9 aptitudes: General 

Learning Ability, Verbal Aptitude, Numerical Aptitude, Spatial 
Aptitude, Form Perception, Clerical Perception, Motor Coordination, 
Finger Dexterity, and Manual Dexterity. The aptitude scores are 
standard scores with 100 as the average for the general working 
population, with a standard deviation of 20. 

Occupational norms are established in terms of minimum qualifying 
scores for each of the significant aptitude measures which, in 
combination, predict job performance. For any given occupation, 
cutting scores are set only for those aptitudes which contribute 
to the prediction of performance of the job duties of the experi- 
mental sample. It is important to recognize that another job might 
have the same job title but the job content might not be similar. 

The GATB norms described in this report are appropriate for use 
only for jobs with content similar to that shown in the job descrip- 
tion included in this report. 



Charles E. Odell, Director 
U. S . Employment Service 
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GATB Study #2447, 2691 



DEVELOPMENT OF USES APTITUDE TEST BATTERY 

FOR 

Programmer, Business (profess. 6 kin.) 020.188.026 

S-314 

This report describes research undertaken for the purpose of validating and 
crosswalidating General Aptitude Test Battery (GATB) norms for the occupation 
of Programmer, Business (profess. 6 kin.) 020.188.026. The following norms 
were established: 

. «... . Minimum Acceptable 

GATB Aptitudes GATB Scores 

G - General Learning Ability 115 

V - Verbal Aptitude 105 

N - Numerical Aptitude 110 

S - Spatial Aptitude 105 



RESEARCH SUMMARY - VALIDATION SAMPLE 



' Sample : 

102 ( 82 male and 20 female) workers employed as Business Programmers at 
various establishments in M^lwqukee, Wisconsin affiliated with the Data 
Processing Management Association. 

Criterion : 

Supervisory ratings 

Design ; 

Concurrent (test and criterion data were collected at approximately the 
same time.) 

Minimum aptitude requirements were determined on the basis of a job 
analysis and statistical analyses of aptitude mean scores, standard 
deviations, and selective efficiencies. 

Concurrent Validity : 

Phi Coefficient = .33 (P/2 less than .0065) 
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Effectiveness of Norma: 

Only 69% of the nontest-selected workers used for this study were good 
workers; if the workers had been test -selected with the S-314 norms* 79% 
would have been good workers. 31% of the nontest -selected workers used 
for this study were poor workers; if the workers had been test -selected 
with the S-314 norms* only 21% would have been poor workers. The 
effectiveness of the norms is shown graphically in Table 1: 

' TABLE 1 

Effectiveness of Norms 

Without Tests With Tests 

Good Workers 69% 79% 

Poor Workers 31% 21% 

VALIDATION SAMPLE DESCRIPTION 



Size : 

N = 102 

Occupational Status : 

Employed workers 

Work Setting : 

Allis-Chalmers Manufacturing Campany 
Badger Mutual Insurance Company 
First Wisconsin National Bank 
Beil Company 

Marine National Exchange Bank 

Marshall and Ilsley Bank 

George J. Meyer Manufacturing Company 

Miller Brewing Company 

Milwaukee Gas Light Company 

Mobil Oil Company 

Northwestern Mutual Life Insurance Company 
Joseph Schlits Brewing Company 
A. 0. Smith Corporation 
West Bend Aluminum Company 
Wisconsin Electric Power Company 
Wisconsin Telephone Company 

Employer Selection Requirements : 

Education: High School graduation 

Previous Experience: Completion of one month IBM programmer training course. 

Tests : IBM Revised Programmer Aptitude Test 

Principal Activities : 

The job duties for each worker are comparable to those shown in the job 
description on the Fact Sheet in the Appendix. 

O Minimum Exoerience: 

'Ail worxers in the sample had at least one month total job experience. 
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TABLE 2 

Means* Standard Deviations* Ranges and Pearson Product -Moment Correlations with 



the Criterion 


(r) for Age, 


Education * 


and Experience 






Mean 


SD 


Range 


r 


Age (years) 


29.7 


6.7 


18-51 


-.104 


Education (years) 


14.4 


1.8 


12-18 


.176 


Experience (months) 


25.5 


21.3 


1-90 


.007 



EXPERIMENTAL TEST BATTERY 

All 12 tests of the GATB, B-1002B were administered during the period of 
April 1962 to April 1963. 



CRITERION 

The criterion data consisted of supervisory ratings of job proficiency made at 
approximately the same time as test data were collected. The worker’s immediate 
supervisor made two ratings with a time interval of at least three weeks between 
ratings. 

Rating Scale: 

USES Form SP-21 "Descriptive Rating Scale." (See Appendix.) This scale 
consists of nine items covering different aspects of job performance. Each 
item has five alternatives corresponding to different degrees of job profi- 
ciency. 

Reliability: 

fne coefficient of reliability between the two ratings is .96 indicating a 
significant relationship. Therefore* the final criterion consisted of the 
combined scores of the two sets of ratings. 

Criterion Score Distribution : 

Possible Range: 18-90 

Actual Range: 38-90 

Mean : 70.0 

Standard Deviation: 11.0 



Criterion Dichotomy : 

The criterion distribution was dichotomized into low and high groups by 
placing 31% of the sample in the low group to correspond with the 
percentage of workers considered unsatisfactory or marginal. Workers 
in the high criterion group were designated as "good workers" and those 
in the low group as "poor workers." The criterion critical score is 66. 
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APTITUDES CONSIDERED FOR INCLUSION IN THE NORMS 



Aptitudes were selected for tryout in the norms on the basis of a qualitative 
analysis of job duties involved and a statistical analysis of test and criterion 
data* Aptitudes V and Q which do not have a high correlation with the criterion 
were considered for inclusion in the norms because the qualitative analysis 
indicated that these aptitudes were important for the job duties and the sample 
had a relatively high mean score on aptitude Q end a relatively low standard 
deviation on aptitude V. With employed workers a relatively high mean score 
may indicate that some sample pre-selection has taken place* Tables 3, 4 and 

5 show the results of the qualitative and statistical analyses* 

TABLE 3 

Qualitative Analysis 

(Based on the job analysis the aptitudes indicated 
appear to be important to the work performed) 

Aptitude Rationale 

6 - General Learning Ability Required to understand and apply work 

statement instructions , recommended procedural 
routines, and related informational data; to 
identify and organize elements of a problem 
into logical sequence for computer operation 
by means of preparing block diagrams and flow 
charts. Required to make analytical and logical 
analyses in planning procedural routines; to 
have a working knowledge of the company business 
organization and management and with modem 
office methods and procedures; and to have a 
complete familiarity with programming principles 
and techniques in order to discuss programing 
methods, requirements, and approaches with line and 
staff personnel* 

V - Verbal Aptitude Required to read and understand work statements, 

procedural routines, and related data; to give 
and exchange information pertinent to programming 
techniques and principles while attending briefings, 
meetings, and interviews; and to document programs 
and prepare accurate and complete reports* 

N - Numerical Aptitude Required to make an analytical and logical 

analysis of mathematical problems, resolve compu- 
tations, and arrange computations into proper 
program sequence for computer operations* 

S - Spatial Aptitude Required to mentally visualize flow of data 

through computer system. Required to interpret 
and develop diagrams and flow charts in proper 
relationships to obtain desired final printed 
results. 



Q - Clerical Perception 
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Required to perceive pertinent detail in program 
documentation | assembled data, and recommended 
program routines; to prepare input, output, and 
nomenclature lists; and to translate step by 
step instructions from flow chart for the console 
operator. Required to recognize and detect 
errors in program instructions, to correct errors 
by altering sequence of flow chart steps, and to 
avoid perceptual errors in making computations. 



TABLE 4 

Means, Standard Deviations C9D), Ranges and Pearson Product-Moment Correlations 
with the Criterion (r) for the Aptitudes erf the GATB; N=102 



Aptitudes 


Mean 


SD 


Range 


r 


G - General Learning Ability 


132.1 


12.4 


102-159 


.359** 


V - Verbal Aptitude 


124.8 


13.2 


94-160 


• 046 


N - Numerical Aptitude 


130.8 


13.7 


95-157 


.396** 


S - Spatial Aptitude 


121.6 


16.1 


81-166 


.238* 


P - Form Perception 


120.4 


15.6 


84-157 


,128 


Q - Clerical Perception 


127.7 


15,7 


88-167 


,175 


K - Motor Coordination 


117.4 


14.0 


86-151 


.175 


F - Finger Dexterity 


109.3 


19.1 


66-164 


.013 


M - Manual Dexterity 


112.6 


20.6 


56-158 


.281** 






♦Significant at the 


• 05 level 






♦♦Significant at the 


.01 level 




TABLE 5 



Summary of Qualitative and Quantitative Data 



Type of Evidence 


Aptitudes 


G 


V 


N 


s 


p 


Q 


K 


F 


M 


Job Analysis Data 
Important 


X 


X 


X 


X 




X 








Irrelevant 




















Relatively High Mean 


X 




X 






X 








Relatively Low Standard Dev. 


X 


X 


X 








X 






Significant Correlation 
with Criterion 


X 




X 


X 










X 


Aptitudes to be Considered 
for Trial Norms 


G 


V 


N 


s 




_9_ 






M 
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DERIVATION AND VALIDITY OF NORMS 

Final norms wsre derived on the basis of a comparison of the degree to which 
trial norms consisting of various combinations of aptitudes G, V, N, S, Q and 
M at trial cutting scores were able to differentiate between the 69% of the sample 
considered good workers and the 31% of the sample considered poor workers. Trial 
cutting scores at five-point intervals approximately one standard deviation below 
the mean are tried because this will eliminate about one-third of the sample with 
three-aptitude norms. For two-aptitude trial norms* minimum cutting scores 
slightly more thaq one standard deviation below the mean will eliminate about 
one-third of the sample; for four-aptitude trial norms* cutting scores slightly 
less than one standard deviation below the mean will eliminate about one-third 
of the sample. The Phi Coefficient was used as a basis for comparing trial 
norms. The optimum differentiation for the occupation of Programmer, Business 
020. 188.026 was provided by the norms of G-115, V-105, N-110 and S-105. The 
validity of these norms is shown in Table 6 and is indicated by a Phi Coefficient 
of .33 (statistically significant at the .0005 level). 

TABLE 6 



Concurrent Validity of Test Norms 
G-115, V-105, N-110, S-105 





Nonqualifying 
Test Scores 


Qualifying 
Test Scores 


Total 


Good Workers 


13 


57 


70 


Poor Workers 


17 


15 


32 


Total 


30 


72 


102 



Phi Coefficient * .33 Chi Square (X^y) = 11.0 

Significance Level s P/2 less than .0005 



DETERMINATION OF OCCUPATIONAL APTITUDE PATTERN 

The data for this study met the requirements for incorporating the occupation 
studied into OAP-1 which is shown in Section II of the Manual for the General 
Aptitude Test Battery. A Phi Coefficient of .26 is obtained with the OAP-1 
norms 6-125, N-115 and S-115. 
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CHECK STUDY RESEARCH SUMMARY SHEET 

S-314 GATB #2447 

Programmer, Business (profess. & kin.) 020.188-026 
Cheek Study #1 Research Summary 
Sample. 

93 (77 male and II female) workers employed as Business Programmers 
in California. 



TABLE 7 

Means, Standard Deviations (SD), Ranges, and Pearson Product-Moment Correlations 
with the Criterion (r) for Age, Education, Experience and the 
Aptitudes of the GATB-Cross-Validttion Sample #1 



Age (years) 


34.5 


7.8 


22-53 


-.203 


Education (years) 


14.2 


2.1 


10-20 


.139 


Experience (months) 


30.9 


19.6 


6-108 


-.051 


G - General Learning Ability 


128.7 


16.1 


91-157 


.371** 


V - Verbal Aptitude 


124.5 


15.6 


90-170 


.313** 


N - Numerical Aptitude 


124.6 


15.6 


91-161 


.347** 


S - Spatial Aptitude 


121.6 


15.0 


84-156 


.247* 


P - Form Perception 


117.8 


15.7 


83-163 


.222* 


Q - Clerical Perception 


126.1 


18.3 


87-194 


.299** 


K - Motor Coordination 


119.3 


16.5 


74-151 


.029 


F - Finger Dexterity 


106.7 


17.7 


57-158 


-.009 


M - Manual Dexterity 


112.7 


20.8 


57-154 


-.019 






♦Significant 


at the 


.05 level 


Design t 




♦♦Significant 


at the 


. 01 level 



Concurrent (test and criterion data were collected at approximately the 
same time. ) 

Principal Activities : 

The job duties for each worker are comparable to those shown in 
the job description in the /ppendix. 



Concurrent Validity : 

Phi Coefficient = .29 (P/2^.005) 
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Effectiveness of Norms 



Only 67% of the nontest-selected workers used for this study were good 
workers; if the workers had been test-selected with the S-314 norms, 77% 
would have been good workers* 33% of the nontest-selected workers used 
for this study were poor workers 4 if the workers had been test-selected 
with the S-314 norms, only 23% would have been poor workers. The 
effectiveness of the norms is shown graphically in Table 8 : 

TABLE 8 

Effectiveness of S-314 Norms on Check Study Sample #1 

Without Tests With Tests 

67% 77% 

33% 23% 



TABLE 9 



Concurrent Validity of S-314 Norms on Check Study Sample #1 





Nonqualifying 
Test Scores 


Qualifying 
Test Scores 


Total 


Good Workers 


13 


49 


62 


Poor Workers 


16 


15 


31 


Total 


29 


64 


93 



Phi Coefficient (0) = .29 Chi Square (X 2 ) =7.7 

Significance Level = P/2 <.005 



Good Workers 
Poor Workers 



S-314 
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6ATB #2691 



Progaammer, Business (profess. 6 kin.) 020.188-026 
Check Study #2 Research Summary 
Sample ; 

3 

62 workers (56 male 6 6 female) employed as Business Programmers by 
various firms and governmental agencies in Ohio. 

TABLE 10 

Means, Standard Deviations (SD), Ranges and Pearson-Product Moment Correlations 

with the Criterion (r) for Age, Education, Experience and the Aptitudes of the GATB- 
Cross- Validation Sample #2 





Mean 


SO 


Range 


t 


Age (years) 


32.3 


5.6 


25- 52 


-.192 


Education (years) 


15.9 


.8 


14- 18 


.174 


Experience (months) 


50.9 


20.9 


24-132 


-.028 


G - General Learning Ability 


126.0 


13.4 


93-154 


.276* 


V - Verbal Aptitude 


120.9 


14.1 


92-151 


.128 


N - Numerical Aptitude 


123.5 


16.3 


74-153 


.246 


S - Spatial Aptitude 


122.7 


15.7 


91-166 


. 337** 


P - Form Perception 


113.7 


17.5 


76-156 


.118 


Q - Clerical Perception 


119.7 


15.9 


81-152 


.221 


K - Motor Coordination 


111.1 


16.4 


78-153 


.017 


F - Finger Dexterity 


99.7 


18.0 


55-135 


.322* 


M - Manual Dexterity 


99.5 


24.1 


51-159 


.181 



♦Significant at the .05 level 
♦♦Significant at the .01 level 

Criterion : 

Supervisory ratings 
Design : 

Concurrent (test and criterion data were collected at approximately 
the same time). 

Principal Activities: 



The duties for this sample are comparable to those shown in tha job 
description in the Appendix. 

Concurrent Validity : 

Phi Coefficient (0) = .23 (P/2 <- 05) 

Effectiveness of Norms: 



ERIC 



Oily 65% of the nontest-selected workers ised for this study were good 
workers; if the workers had been test-selected with the S-314 norms, 74% 
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would have been good workers. 35% of the nontest -selected workers used 
for this study were poor workers; if the workers had been test -selected 
with S-314 norms only 26% would have been poor workers. The effectiveness 
of the norms is shown graphically in Table 11. 

TABLE 11 

Effectiveness of S-314 Norms en 
Check Study Sample § 2 



Good Workers 65% 74% 

Poor Workess 35% 26% 



TABLE 12 



Concurrent Validity of S-314 Norms 
(G-115, V-105, N-110, S-105) 
Check Study #2 





Nonqualifying 


Qualifying 






Test Scores 


Test Scores 


Total 


Good Workers 


11 


29 


40 


Poor Workers 


12 


10 


22 


Total 


23 


39 


62 



Phi Coefficient - .23 Chi Square (X 2 y) s 3.4 

Significance Level = P/2 ^05 
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DESCRIPTIVE BATING SCALE 
(For Aptitude Test Development Studies) 



Score 



RATING SCALE FOR 



D. O. T. Title and Code 



Directions} Please read Form SP-20, "Suggestions to Raters", and then fill in 
the items listed below. In making your ratings, only one box 
should be checked for each question. 

Name of Worker (print) 

(Last) (First) 



Sex: Mal e Female. 

Company Job Title: 



How often do you see this worker in a work situation? 

/ / See him at work all the time. 

/ / See him at work several times a day. 

/ / See him at work several times a week. 

/" 7 Seldom see him in work situation. 

How long have you worked with him? 

/ ' / Under one month* 

/ / One to two months* 

/ / Three to five months. 

/ / Six months or more. 



O 
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A. How much work can he get done? (Worker's ability to make efficient use of 

his time and to work at high speed.) 

£7i- Capable of very low work output. Can perform only at an unsatis- 
factory pace. 

/ 7 2. Capable of low work output. Can perform at a slow pace. 

/ ' 7 3. Capable of fair work output. Can perform at an acceptable but not 
a fast pace. 

£7 Capable of high work output. Can perform at a fast pace. 

£7 5. Capable of very high work output. Can perform at an unusually fast 
pace. 

B. How good is the quality of his work? (Worker's ability to do high-grade work 

which meets quality standards.) 

£7 1 . Performance is inferior and almost never meets minimum quality 
standards . 

£72- The grade of his work could stand improvement. Performance is usually 
acceptable but somewhat inferior in quality. 

/ 7 3. Performance is acceptable but usually not superior in quality. 

/' '/ It. Performance is usually superior in quality* 

05. Performance is almost always of the highest quality. 



C. How accurate is he in his work? (Worker's ability to avoid making mistakes.) 
/ / 1. Makes very many mistakes. Work needs constant checking. 

/ / 2. Makes frequent mistakes. Work needs more checking than is desirable. 
/ 7 3. Makes mistakes occasionally. Work needs only normal checking. 

/ j It. Makes few mistakes. Work seldom needs checking. 

OS. Rarely makes a mistake. Work almost never needs checking. 
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D. 



How much does he know about his job? (Worker's understanding of the principles, 
equipment, materials and methods that have to do directly or indirectly with his 
work. ) 



□ 1 . 
cm 2. 

cm 3. 
cm 4. 
cm 5. 



Has very limited knowledge. Does not know enough to do his job 
adequately. 

Has little knowledge. Knows enough to "get by." 

Has moderate amount of knowledge. Knows enough to do fair work. 
Has broad knowledge. Knows enough to do good work. 

Has complete knowledge. Knows his job thoroughly. 



E. How much aptitude or facility does he have for this kind of work? (Worker's 
adeptness or knack for performing his job easily and well.) 

□ i. Has great difficulty doing his job. Not at all suited to this kind 
of work. 



□ 2. Usually has some difficulty doing his job. Not too well suited to 

this kind of work. 

□ 3. Does his job without too much difficulty. Fairly well suited to this 

kind of work. 

□ 4. Usually does his job without difficulty. Well suited to this kind 

of work. 



□ 5. 



Does his job with great ease. Exceptionally well suited for this 
kind of work. 



F. How large a variety of job duties can he perform efficiently? (Worker's 
ability to handle several different operations in his work.) 



□ i. 

□ 2 . 

□ 3. 

□ 4. 

□ 5. 



Cannot perform different operations adequately. 

Can perform a limited number of different operations efficiently. 

Can perform several different operations with reasonable efficiency. 

Can perform many different operations efficiently. 

Can perform an unusually large variety of different operations 
efficiently. 
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G, How resourceful is he when something different comes up or something out of 
the ordinary occurs? (Worker's ability to apply what he already knows to a 
new situation.) 



/ / l. Almost never is able to figure out what to do. Needs help on eveii 
minor problems. 

n*- Often has difficulty handling new situations. Needs help on all but 

simple problems, 

O 3. Sometimes knows what to do, sometimes doesn't. Can deal with problems 
that are not too complex. 

Dk. Usually able to handle new situations. Needs help on only complex 
problems. 

0 5. Practically always figures out what to do himself. Rarely needs 
help, even on complex problems. 

H. How many practical suggestions does he make for doing things in better ways? 

(Worker's ability to improve work methods.) 



O 1. Sticks strictly with the routine. Contributes nothing in the way 
of practical suggestions. 

/ / 2. Slow to see new ways to improve methods. Contributes few practical 
suggestions . 

£7 3- Neither quick nor slow to see new ways to improve methods. Contributes 
some practical suggestions. 

/ / U, Quick to see new ways to improve methods. Contributes more than his 
share of practical suggestions. 



£7 5 . Extremely alert to see new ways to improve methods. Contributes an 
unusually large number of practical suggestions. 




I. Considering all the factors already rated, and only these factors, how acceptable 
is his work? (Worker's "all-around" ability to do his job.) 

£7 1. Would be better off without him. Performance usually not acceptable. 

£7 2. Of limited value to the organization. Performance somewhat inferior. 

£7 3. A fairly proficient worker. Performance generally acceptable. 

L 7 U. A valuable worker. Performance usually superior. 

/ / 5. An unusually competent worker. Performance almost always top notch. 
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S-314 



FACT SHEET 

Job Title: Programmer, Business (profess. 6 kin.) 020.188-026 

Job Summary: Develops and prepares diagrammatic plans and written instructions 

for solution of business problems by means of automatic data-processing equip- 
ment. 

Work Performed : Converts symbolic statement of business problems to detailed 

logical flow charts for coding into computer language. Analyzes all or part 
of work flow chart or diagram representing business problem by applying 
knowledge of computer capabilities, subject matter, algebra and symbolic 
logic to develop sequence of program steps. Confers with supervisor and 
representatives of departments affected by program, to resolve questions of 
program intent, output requirements, input data acquisition, extent of 
automatic programming and coding use and modification, and inclusion of internal 
checks and controls. Writes detailed logical flow chart in symbolic form 
to represent work ordeb of data to be processed by the computer system, and 
describe input, output, arithmetic and logical operations involved. Converts 
detail logical flow chart to language processable by computer. Devises sample 
input data to provide test of program adequacy. Prepares block diagrams to 
specify equipment configuration. Observes or runs tests of coded program on 
computer using actual or sample input data. Corrects program errors by such 
methods as altering program steps and sequence. Prepares written instruction 
(run book) to guide operating personnel during production runs. Analyzes, reviews, 
and rewrites programs to increase operating efficiency or adapt to new requirements. 
Compiles documentation of program development and subsequent revisions. May 
specialize in writing programs for one make and type of computer. 

Effectiveness of Norms * Only 69% of the nontest-selected workers used for this 
study were good workers; if the workers had been test-selected with the S-314 
norms, 79% would have been good workers. 31% of the non test -selected workers 
used for this study were poor workers; if the workers had been test -selected 
with the S-314 norms, only 21% would have been poor workers. 

Applicability of S-314 Norms : The aptitude test battery is applicable to jobs 

wnlcn include a majority of duties described above. 
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